Skip to content

feat: add support for lhs compiled nodes - #26

Open
k13gomez wants to merge 4 commits into
mainfrom
lhs-compile
Open

k13gomez wants to merge 4 commits into
mainfrom
lhs-compile

Conversation

@k13gomez

Copy link
Copy Markdown
Member
  • Compile LHS expressions at defrule/defquery load time and use them in place of eval in mk-session.
  • Use the precompiled LHS and RHS fns when deserializing a rulebase, and keep the eval fallback for older data.
  • Fix deserialization with :compiler-cache false when an expression still needs eval.
  • Look up named queries by name, so query vars work with rulebases from earlier versions.

This is the Clara rules compiler, translating raw data structures into compiled versions and functions.
Most users should use only the clara.rules namespace."
(:require [clara.rules.engine :as eng]
[clara.rules.accumulators] ;;; for exists

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a small change because exists gets re-written as an accumulator below

alpha-graph :- [schema/AlphaNode]]
(let [backward-edges (:backward-edges beta-graph)
handle-expr (fn [id->expr s-expr id expr-key compilation-ctx]
handle-expr (fn [id->expr s-expr id expr-node-type compilation-ctx]

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you will notice I renamed expr-key to expr-node-type because it better expresses what the key is, perhaps? I can then repurpose expr-key to mean something else for indexing precompiled expressions

id
:test-expr
{:compile-ctx {:condition condition
{:expr-key (expr-key :test-expr (:constraints condition) (env-keys (:env beta-node)))

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the expr-key will be serialized along with the handler var, so that during deserialization we can resolve the handler (if it exists in the classpath)

(assoc ~rule :handler '~handler-name))
([~@(take 1 rule-args)]
(~rule-name '?__token__ {}))
(~rule-name ~'?__token__ {}))

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mrrodriguez this is the fix we had discussed yesterday

@mrrodriguez mrrodriguez Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is OK, but you could also just do (first rule-args) to avoid the need to have to know what it actually is and not get it wrong again.


(declare add-production)

(sc/defn ^:private get-complex-negation :- (sc/maybe

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the compiled rule lookup would always eval the rule generated here

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, this one will always be eval'd, there is no way to AOT compile this one currently, and that is a known trade off for now.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is that obvious to me why this is the case. Is it because there is not a symbol to resolve for it? This is actually related to A brighter concern where I think it is odd that you have to have a resolvable symbol to represent this when it seems like it could just be data directly on the production data structure instead of needing that. It just makes the data oriented API more awkward to work with. If you wanted to have the same precompiled LHS and some other data structure builder, now you would have to also define a var to its expressions.

What is the reason we need that?

@mrrodriguez mrrodriguez left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have only scanned through this at a high level at this point, but I will drop the comments that I do have for now. I have not really looked at the more detailed parts of the mechanics of this.

:beta-children (distinct node-ids)}
(seq env) (assoc :env env))))))

(defn build-lhs-exprs

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder how much slower it is to do this one rule at a time rather than in batch like it is done until this change?

(defn- query*
[rulebase memory query params]
(let [query-node (get-in rulebase [:query-nodes query])]
(let [query-node (get-in rulebase [:query-nodes (or (:name query) query)])] ;;; lookup by name or value

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a little bit weird because if the query actually has changed, it could even have different parameters now. However, you would be resolving it to whatever it used to be. So the call one would not really align anymore.

Comment thread CHANGELOG.md
* A fn is matched by the condition data that its expression is built from. Rules built at runtime still use `eval`. A complex negation still uses `eval` for its generated condition.
* Deserialization of a rulebase uses the precompiled LHS and RHS fns when the rule namespaces are on the classpath. It loads those namespaces when needed. Otherwise it uses `eval`, as before.
* The serialized expression data now holds the `:handler` var symbol and the `:expr-key` of each precompiled expression. A rulebase serialized by an earlier version still deserializes with `eval`.
* Fix deserialization with `:compiler-cache false` when an expression still needs `eval`. Before, a `:cache-key` stored in the serialized data made it throw a `ClassCastException`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find this sentence to be confusing. Why is it holding this key still and why does it get a class-cast exception?

Comment thread CHANGELOG.md

# 1.8.0
* `defrule` and `defquery` now compile their left-hand-side (LHS) expressions when the form is evaluated. An LHS that does not compile now fails at load time, not at `mk-session`.
* The compiled LHS fns are stored in the `:clara.rules/exprs` metadata of the rule or query var. `mk-session` uses them in place of `eval`. A node that several rules share uses one fn.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why metadata instead of first class data on the data structure?


(declare add-production)

(sc/defn ^:private get-complex-negation :- (sc/maybe

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is that obvious to me why this is the case. Is it because there is not a symbol to resolve for it? This is actually related to A brighter concern where I think it is odd that you have to have a resolvable symbol to represent this when it seems like it could just be data directly on the production data structure instead of needing that. It just makes the data oriented API more awkward to work with. If you wanted to have the same precompiled LHS and some other data structure builder, now you would have to also define a var to its expressions.

What is the reason we need that?

(s/optional-key :props) {s/Keyword s/Any}
(s/optional-key :env) {s/Keyword s/Any}
(s/optional-key :handler) (s/maybe Function)
(s/optional-key :handler) (s/maybe s/Symbol)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was just wrong before? The code had this a symbol prior to these changes from what I can see.

(defquery temperature-below-value-using-keyword-arg
[:?value]
[:temperature [{value :value}] (< value ?value)])
[:temperature [{value :value}] (= ?value value)])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Query no longer has name somantically for what it is doing. Why was the equals changed here?

(let [id-counter (atom 0)
create-id-fn (fn [] (swap! id-counter inc))
beta-graph (to-beta-graph #{production} create-id-fn)
alpha-graph (to-alpha-graph beta-graph create-id-fn)]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part worries me in that the compiled form will build into assumptions here about how the "whole network" is compiled later in how it knows how to to compile the expressions here.

In a same-runtime scenario, it would not differ. However, with serialization it could. I think the compiled expressions may need some sort of "version" associated with them so if the network building logic changes later, those compiled forms are invalidated and eval'ed JIT instead of using the AOT compiled form.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants